Information providing system

ABSTRACT

The invention provides a novel information providing system as well as a mobile information terminal and an information processing device which are used for the information providing system. In the information providing system, information to be registered in a database is provided from a mobile information terminal to an information processing device. The mobile information terminal acquires speech information based on speech inputted thereto as registration information during an information registration mode, also acquires position information of the mobile information terminal as registration information during the information registration mode, and accepts the inputs of additional information relevant to the speech information and the position information as registration information during the information registration mode and then transmits these information to the information processing device, while the information processing device receives the speech information, the position information and the additional information sent from the mobile information terminal and stores the received information in the database, relating them to each other. Thereby, the database registration and database search of various information such as city information can be carried out easily and accurately by anyone with the new information providing system.

TECHNICAL FIELD

The present invention relates to an information providing system, moreparticularly to a novel information providing system capable of easilyperforming the database registration and database search of variousinformation such as city information, and a mobile information terminaland an information processing device used for the information providingsystem.

BACKGROUND ART

In general, the system for the database registration and database searchof various information such as city information has its data in the formof computerized drawing data or letter-symbol data like a map on a carnavigation system, a map on the Internet and such.

However, the information computerization needs to be accuratelyperformed by a skilled person accustomed to manipulation ofcomputerization devices, for example an input device of a keyboard ormouse, an image capture device, a map load program, etc. Thus, noteverybody is capable of easily handling the computerization and databaseregistration. This can be similarly said about the city information.

Also, the database search requires, for example, keyboard inputmanipulation on a computer, and is inconvenient to a person who isunaccustomed to such manipulations.

Nowadays, a mobile terminal such as a cellular phone has become capableto retrieve and obtain city information. However, its significantlysmall input keys do not provide easy manipulation for an unskilledperson and even for a skilled person, which force a longer operationtime.

Considering the above, a registration or retrieval method with voiceinput might be employed. But, the prior-art speech recognitiontechnology has poor accuracy of vocabulary data conversion regardingnon-dictionary vocabularies such as proper nouns, coined words,abbreviated words, colloquial words, etc., and in particular, cityinformation in which proper names are frequently used cannot berecognized with favorable accuracy.

In view of the above, the present invention has an object to provide anovel information providing system capable to make the databaseregistration and database search of various information including cityinformation easy and highly-accurate for anybody, and another object toprovide a mobile information terminal and an information processingdevice which are used for the information providing system.

DISCLOSURE OF INVENTION

In order to accomplish the object, the present invention first providesan information providing system in which information to be registered ina database is provided from a mobile information terminal to aninformation processing device, characterized in that said mobileinformation terminal comprising a speech input unit capable of inputtingspeech as registration information during an information registrationmode; a speech processing unit capable of acquiring speech informationbased on the speech inputted by the speech input unit; a positionverification unit capable of acquiring position information of saidmobile information terminal as registration information during theinformation registration mode; an additional information input unitcapable of inputting additional information relevant to the speechinformation and the position information as registration informationduring the information registration mode; and a transmitter unit capableof transmitting to the information processing device, the speechinformation obtained by the speech processing unit, the positioninformation obtained by the position verification unit and theadditional information inputted by the additional information inputunit; and also characterized in that said information processing devicecomprising a receiver unit capable of receiving the speech information,the position information and the additional information sent from saidmobile information terminal; and an information processing unit capableof mutually relating and storing the speech information, the positioninformation and the additional information received by the receiverunit, in the database.

Secondly, the invention provides an information providing system inwhich information registered in a database is provided from aninformation processing device to a mobile information terminal havingmade a request for information provision, characterized in that saidmobile information terminal comprising a speech input unit capable ofinputting speech relevant to requested information as a search keyduring an information provision request mode; a speech processing unitcapable of acquiring speech information based on the speech inputted bythe speech input unit; a position verification unit capable of acquiringposition information of said mobile information terminal as a search keyduring the information provision request mode; a transmitter unitcapable of transmitting to the information processing device, the speechinformation obtained by the speech processing unit and the positioninformation obtained by the position verification unit; and a receiverunit capable of receiving additional information sent from theinformation processing device, as the requested information; and alsocharacterized in that said information processing device comprising areceiver unit capable of receiving the speech information and theposition information sent from said mobile information terminal; aninformation processing unit capable of retrieving the additionalinformation relevant to the speech information and the positioninformation received by the receiver unit, from the database havingspeech information, position information and additional informationmutually associated and prestored therein; and a transmitter unitcapable of transmitting the additional information retrieved by theinformation processing unit, to said mobile information terminal.

Thirdly, the invention provides an information providing system in whichinformation to be registered in a database is provided from a mobileinformation terminal to an information processing device and informationregistered in the database is provided from the information processingdevice to a mobile information terminal having made a request forinformation provision, characterized in that said mobile informationterminal comprising a speech input unit capable of inputting speech asregistration information during an information registration mode, andcapable of inputting speech relevant to requested information as asearch key during an information provision request mode; a speechprocessing unit capable of acquiring speech information based on thespeech inputted by the speech input unit during the informationregistration mode, and capable of acquiring speech information based onthe speech inputted by the speech input unit during the informationprovision request mode; a position verification unit capable ofacquiring position information of said mobile information terminal asregistration information during the information registration mode, andcapable of acquiring position information of said mobile informationterminal as a search key during the information provision request mode;an additional information input unit capable of inputting additionalinformation relevant to the speech information and the positioninformation as registration information during the informationregistration mode; a transmitter unit capable of transmitting the speechinformation obtained by the speech processing unit, the positioninformation obtained by the position verification unit and theadditional information inputted by the additional information inputunit, to the information processing device during the informationregistration mode, and capable of transmitting the speech informationobtained by the speech processing unit and the position informationobtained by the position verification unit, to the informationprocessing device during the information provision request mode; and areceiver unit capable of receiving additional information sent from theinformation processing device, as the requested information; and alsocharacterized in that said information processing device comprising areceiver unit capable of receiving the speech information, the positioninformation and the additional information sent from said mobileinformation terminal during the information registration mode, andcapable of receiving the speech information and the position informationsent from said mobile information terminal during the informationprovision request mode; an information processing unit capable ofmutually relating and storing the speech information, the positioninformation and the additional information received by the receiver unitduring the information registration mode, in the database, and capableof retrieving the additional information relevant to the speechinformation and the position information received by the receiver unit,from the database, during the information provision request mode; and atransmitter unit capable of transmitting the additional informationretrieved by the information processing unit, to said mobile informationterminal.

Further, in order to accomplish the object, the invention fourthlyprovides an information providing system in which information to beregistered in a database is provided from a mobile information terminalto an information processing device, characterized in that said mobileinformation terminal comprising a speech input unit capable of inputtingspeech as registration information during an information registrationmode; a speech processing unit capable of acquiring speech informationbased on the speech inputted by the speech input unit; a positionverification unit capable of acquiring position information of saidmobile information terminal as registration information during theinformation registration mode; an image capture unit capable ofcapturing an image as registration information during the informationregistration mode; an image processing unit capable of acquiring imageinformation based on the image captured by the image capture unit; anadditional information input unit capable of inputting additionalinformation relevant to the speech information, the position informationand the image information as registration information during theinformation registration mode; and a transmitter unit capable oftransmitting to said information processing device, the speechinformation obtained by the speech processing unit, the positioninformation obtained by the position verification unit, the imageinformation obtained by the image processing unit and the additionalinformation inputted by the additional information input unit; and alsocharacterized in that said information processing device comprising areceiver unit capable of receiving the speech information, the positioninformation, the image information and the additional information sentfrom said mobile information terminal; and an information processingunit capable of associating the speech information, the positioninformation, and the image information and the additional informationreceived by the receiver unit, with one another, and then stores them inthe database.

Fifthly, the invention provides an information providing system in whichinformation registered in a database is provided from an informationproviding device to a mobile information terminal having made a requestfor information provision, characterized in that said mobile informationterminal comprising a speech input unit capable of inputting speechrelevant to requested information as a search key during an informationprovision request mode; a speech processing unit capable of acquiringspeech information based on the speech inputted by the speech inputunit; a position verification unit capable of acquiring positioninformation of said mobile information terminal as a search key duringthe information provision request mode; an image capture unit capable ofcapturing an image as a search key during the information provisionrequest mode; an image processing unit capable of acquiring imageinformation based on the image captured by the image capture unit; atransmitter unit capable of transmitting to said information processingdevice, the speech information obtained by the speech processing unit,the position information obtained by the position verification unit andthe image information obtained by the image processing unit; and areceiver unit capable of receiving additional information sent from saidinformation processing device, as the requested information; and alsocharacterized in that said information processing device comprising areceiver unit capable of receiving the speech information, the positioninformation and the image information sent from said mobile informationterminal; an information processing unit capable of retrieving theadditional information relevant to the speech information, the positioninformation and the image information received by the receiver unit,from the database having speech information, position information, imageinformation and additional information mutually associated and prestoredtherein; and a transmitter unit capable of transmitting the additionalinformation retrieved by the information processing unit, to said mobileinformation terminal.

Sixthly, the invention provides an information providing system in whichinformation to be registered in a database is provided from a mobileinformation terminal to an information processing device and informationregistered in the database is provided from said information processingdevice to a mobile information terminal having made a request forinformation provision, characterized in that said mobile informationterminal comprising a speech input unit capable of inputting speech asregistration information during an information registration mode, andcapable of inputting speech relevant to requested information as asearch key during an information provision request mode; a speechprocessing unit capable of acquiring speech information based on thespeech inputted by the speech input unit during the informationregistration mode, and capable of acquiring speech information based onthe speech inputted by the speech input unit during the informationprovision request mode; a position verification unit capable ofacquiring position information of said mobile information terminal asregistration information during the information registration mode, andcapable of acquiring position information of said mobile informationterminal as a search key during the information provision request mode;an image capture unit capable of capturing an image as registrationinformation during the information registration mode, and capable ofcapturing an image as a search key during the information provisionrequest mode; an image processing unit capable of acquiring imageinformation based on the image captured by the image capture unit duringthe information registration mode, and capable of acquiring imageinformation based on the image captured by the image capture unit duringthe information provision request mode; an additional information inputunit capable of inputting additional information relevant to the speechinformation, the position information and the image information asregistration information during the information registration mode; atransmitter unit capable of transmitting the speech information obtainedby the speech processing unit, the position information obtained by theposition verification unit, the image information obtained by the imageprocessing unit and the additional information inputted by theadditional information input unit, to said information processing deviceduring the information registration mode, and capable of transmittingthe speech information obtained by the speech processing unit, theposition information obtained by the position verification unit and theimage information obtained by the image processing unit, to saidinformation processing device during the information provision requestmode; and a receiver unit capable of receiving additional informationsent from said information processing device, as the requestedinformation; and also characterized in that said information processingdevice comprising a receiver unit capable of receiving the speechinformation, the position information, the image information and theadditional information sent from said mobile information terminal duringthe information registration mode, and capable of receiving the speechinformation, the position information and the image information sentfrom said mobile information terminal during the information provisionrequest mode; an information processing unit capable of mutuallyrelating and storing the speech information, the position information,the image information and the additional information received by thereceiver unit, in the database, during the information registrationmode, in the database, and capable of retrieving the additionalinformation relevant to the speech information, the position informationand the image information received by the receiver unit during theinformation provision request mode; and a transmitter unit capable oftransmitting the additional information retrieved by the informationprocessing unit, to said mobile information terminal.

And, the invention seventhly provides an information providing systemcharacterized in that the speech information is information whichcontains a phoneme string of the speech, and it eighthly provides aninformation providing system characterized in that the image informationis features of the image.

Furthermore, in order to accomplish the object, the invention provides amobile information terminal which provides information to be registeredin a database to an information processing device, comprising a speechinput unit capable of inputting speech as registration informationduring an information registration mode; a speech processing unitcapable of acquiring speech information based on the speech inputted bythe speech input unit; a position verification unit capable of acquiringposition information of said mobile information terminal as registrationinformation during the information registration mode; an additionalinformation input unit capable of inputting additional informationrelevant to the speech information and the position information asregistration information during the information registration mode; and atransmitter unit capable of transmitting the speech information obtainedby the speech processing unit, the position information obtained by theposition verification unit and the additional information inputted bythe additional information input unit, to said information processingdevice capable of mutually relating and storing them in the database; ittenthly provides a mobile information terminal to which informationregistered in a database is provided from an information processingdevice by making a request for information provision, comprising aspeech input unit capable of inputting speech relevant to requestedinformation as a search key during an information provision requestmode; a speech processing unit capable of acquiring speech informationbased on the speech inputted by the speech input unit; a positionverification unit capable of acquiring position information of saidmobile information terminal as a search key during the informationprovision request mode; a transmitter unit capable of transmitting tosaid information processing device, the speech information obtained bythe speech processing unit and the position information obtained by theposition verification unit; and a receiver unit capable of receiving asthe requested information, additional information relevant to the speechinformation and the position information as has been retrieved and sentfrom the database having speech information, position information andadditional information mutually related and prestored therein by saidinformation processing device; it eleventhly provides a mobileinformation terminal which provides information to be registered in adatabase, to an information processing device, comprising a speech inputunit capable of inputting speech as registration information during aninformation registration mode; a speech processing unit capable ofacquiring speech information based on the speech inputted by the speechinput unit; a position verification unit capable of acquiring positioninformation of said mobile information terminal as registrationinformation during the information registration mode; an image captureunit capable of capturing an image as registration information duringthe information registration mode; an image processing unit capable ofacquiring image information based on the image captured by the imagecapture unit; an additional information input unit capable of inputtingadditional information relevant to the speech information, the positioninformation and the image information as registration information duringthe information registration mode; and a transmitter unit capable oftransmitting the speech information obtained by the speech processingunit, the position information obtained by the position verificationunit, the image information obtained by the image processing unit andthe additional information inputted by the additional information inputunit, to said information processing device capable of mutually relatingand storing them in the database; it twelfthly provides a mobileinformation terminal which provides information registered in adatabase, from an information processing device to a mobile informationterminal having made a request for information provision, comprising aspeech input unit capable of inputting speech relevant to requestedinformation as a search key during an information provision requestmode; a speech processing unit capable of acquiring speech informationbased on the speech inputted by the speech input unit; a positionverification unit capable of acquiring position information of saidmobile information terminal as a search key during the informationprovision request mode; an image capture unit capable of capturing animage as a search key during the information provision request mode; animage processing unit capable of acquiring image information based onthe image captured by the image capture unit; a transmitter unit capableof transmitting to said information processing device, the speechinformation obtained by the speech processing unit, the positioninformation obtained by the position verification unit and the imageinformation obtained by the image processing unit; and a receiver unitcapable of receiving as the requested information, additionalinformation relevant to the speech information, the position informationand the image information as has been retrieved and sent from thedatabase having speech information, position information, imageinformation and additional information mutually related and prestoredtherein by said information processing device; it thirteenthly providesa mobile information terminal characterized in that the speechinformation is information which contains a phoneme string of thespeech, and it fourteenthly provides a mobile information terminalcharacterized in that the image information is features of the image.

Still further, in order to accomplish the object, the inventionfifteenthly provides an information processing device which registersinformation provided from a mobile information terminal in a database,comprising a receiver unit capable of receiving, from said mobileinformation terminal, speech information based on speech inputted bysaid mobile information terminal, a position information of said mobileinformation terminal, and additional information relevant to the speechinformation and the position information inputted by said mobileinformation termina; and an information processing unit capable ofmutually relating and storing the speech information, the positioninformation and the additional information received by the receiverunit, in the database; it sixteenthly provides an information processingdevice which provides information registered in a database, to a mobileinformation terminal having made a request for information provision,comprising a receiver unit capable of receiving, from said mobileinformation terminal, speech information based on speech inputted bysaid mobile information terminal, and a position information of saidmobile information terminal; an information processing unit capable ofretrieving additional information relevant to the speech information andthe position information received by the receiver unit, from thedatabase having speech information, position information and additionalinformation mutually related and prestored as registration informationtherein; and a transmitter unit capable of transmitting the additionalinformation retrieved by the information processing unit, to said mobileinformation terminal; it seventeenthly provides an informationprocessing device which registers information provided from a mobileinformation terminal, in a database, comprising a receiver unit capableof receiving, from said mobile information terminal, speech informationbased on speech inputted by said mobile information terminal, a positioninformation of said mobile information terminal, image information basedon an image captured by said mobile information terminal, and additionalinformation relevant to the speech information, the position informationand the image information inputted by said mobile information terminal;and an information processing unit capable of mutually relating andstoring the speech information, the position information, the imageinformation and the additional information received by the receiverunit, in the database; it eighteenthly provides an informationprocessing device which provides information registered in a database,to a mobile information terminal having made a request for informationprovision, comprising a receiver unit capable of receiving, from saidmobile information terminal, speech information based on speech inputtedby said mobile information terminal, a position information of saidmobile information terminal, and image information based on an imagecaptured by said mobile information terminal; an information processingunit capable of retrieving additional information relevant to the speechinformation, the position information and the image information receivedby the receiver unit, from the database having speech information,position information, image information and additional informationmutually related and prestored therein; and a transmitter unit capableof transmitting the additional information retrieved by the informationprocessing unit, to said mobile information terminal; it ninteenthlyprovides an information processing device characterized in that thespeech information is information which contains a phoneme string of thespeech; and it twentiethly provides an information processing devicecharacterized in that the image information is features of the image.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing the flow of the database registration in thepresent invention.

FIG. 2 is a diagram showing the flow of the database search in theinvention.

FIG. 3 is a functional block diagram showing an embodiment of a mobileinformation terminal (terminal station) in the invention.

FIG. 4(A) is a functional block diagram showing an embodiment of aninformation processing device (base station) in the invention, whileFIG. 4(B) is a system configuration diagram showing an embodiment of aninformation providing system in the invention.

FIG. 5 is a more practicable flow chart of a process for the databaseregistration in the invention.

FIG. 6 is a diagram showing data structure examples of a database in theinvention.

FIG. 7 is a more practicable flow chart of a process for the databasesearch in the invention.

REFERENCE NUMERALS AND LETTERS IN THE DRAWINGS DESIGNATE THE FOLLOWING

-   -   A, A′ Mobile information terminals (Terminal stations)    -   10 Antenna    -   11 Control unit    -   15 Input unit    -   15 a Character input unit    -   15 b Speech input unit    -   17 Display unit    -   21 Processing unit    -   21 a Speech feature extraction unit    -   21 b Phoneme string generation unit    -   21 c Image feature extraction unit    -   23 Communication control unit    -   25 Image capture unit    -   27 Position verification unit    -   31 Information synthesis unit    -   33 Storage unit    -   B, B′ Information processing devices (Base stations)    -   40 Antenna    -   41 Communication control unit    -   45 Information processing device    -   47 Information processing unit    -   51 Storage unit    -   53 Speech information storage area    -   55 Position information storage area    -   57 Image information storage area    -   61 Additional information storage area    -   C Network    -   DB, DB′ Databases

BEST MODE FOR CARRYING OUT THE INVENTION

Considerations made by the inventors of the present invention win befirst explained before the description of embodiments of the invention.

In view of a situation where many peculiar proper nouns are usedespecially for place names, etc. and where a conversion efficiency and aconversion accuracy in the case of inputting voice and converting thevoice into a word are inferior and a situation where characters aredifficult to be inputted by keys in a mobile information terminal, theinventors have considered that the efficient information input can berealized by entering city information with a symbol string such asspeech feature or phoneme feature closer to a speech waveform, that is,a “phoneme string”, employed as speech information, rather than byperforming speech recognition of word-level susceptible tomisrecognition.

A “phoneme” signifies a vowel or consonant which is an elementconstituting speech, and “phonemic segments” or “continuous phonemicsegments” may well be considered as the phoneme here. The “phonemicsegments” are elements into which one phoneme has been subdivided. The“continuous phonemic segments” are information which indicates thecontinuous state of the phonemic segments, for example, informationwhich indicates how the elements are connected by taking into accountthe change of the phonemic segments that have been obtained by dividing“A”(Japanese vowel) into its initial end, its middle phase and itsterminal end. And, the “phoneme string” signifies a string in which thephonemes are in a row.

Such a “phoneme string” is employed as speech information, and thisspeech information is provided to an information processing device whichmanages the information registration to a database, the informationretrieval from the database and such, where the speech information isassociated with position information of the position, the direction orthe like of a mobile information terminal at the time when the speechwas uttered or with the image information of the neighborhoodphotographed by an image capture device belonging to the mobileinformation terminal, a separate image capture device capable ofproviding data to the mobile information terminal, or the like. Thus,the information registration and the information retrieval can berealized easily and accurately. Incidentally, on this occasion, the cityinformation to be registered and to be retrieved is statisticallyconverged, whereby the efficient registration of the city informationand a service concerning the browsing of the registered contents can beprovided.

Here, the information processing device is constructed of a devicecapable of registering and retrieving the information, which has, forexample, a main memory unit, a secondary storage unit, a processing unitand a communication unit. As such device, a personal computer, alarge-scale computer or the like can be considered. Further, it is morepreferable that the device is capable of analyzing the information withthe use of a program which statistically analyzes the information storedin the database.

As the mobile information terminal, an information terminal of so-called“portable type” such as, for example, a cellular phone, a PDA (PersonalDigital Assistant), a notebook type computer, a wearable computer, awrist watch type computer and an on-vehicle type computer. Here, themethods for moving, equiping, holding, etc. and further the aspects andshapes and such are not especially restricted.

Between the information processing device and the mobile informationterminal, it is constructed as being capable of exchanging or retrievingthe information each other by a communication network such as theInternet or intranet, via wireless LAN, cellular phone or the like.

The database may be built in the information processing device or maywell be separate from the information processing device. Here, theseparate-type database is constructed so as to be able tobi-directionally communicate with the information processing device byany wireless or wired means.

On the basis of the above considerations, information providingtechnology according to embodiments of the invention will now bedescribed with reference to the drawings.

<<Description of Principles of Registration and Retrieval Processes >>

First, the principles of the data registration and data retrieval of thedatabase in the information providing technology according to theembodiments will be briefly described with reference to FIG. 1 and FIG.2.

<Flow of Principle of Registration Process>

As shown in FIG. 1, first of all, at a step S1, speech waveform andposition information are inputted. At a step S2, speech features areextracted. At a step S3, phoneme string determination is performed byusing evaluation function. Subsequently, at a step S4, the phonemestring and the position information are registered in the database,associating with each other.

<Flow of Principle of Retrieval Process>

As shown in FIG. 2, first of all, at a step S11, speech waveform andposition information are inputted. At a step S12, speech features areextracted. At a step S13, phoneme string determination is performed byusing evaluation function. Subsequently, at a step S14, data in thedatabase is retrieved on the basis of the phoneme string and theposition information. At a step S15, a retrieval result is outputted.

<<Description of Mobile Information Terminal, Information ProcessingDevice, Database and System>>

Now, more practicable examples of the mobile information terminal, theinformation processing device and the database which are used in theregistration process and retrieval process and also the system which isconstituted by them will be described with reference to the drawings.FIG. 3 is a functional block diagram showing an arrangement example ofthe mobile information terminal (terminal station). FIG. 4(A) is afunctional block diagram showing an arrangement example of theinformation processing device (base station), while FIG. 4(B) is aschematic diagram showing an arrangement example of the whole system.

<Mobile Information Terminal>

As shown in FIG. 3, the mobile information terminal A on the user sidehas an antenna 10, a control unit 11 capable of controlling the wholeterminal A, an input unit 15 capable of inputting data, a display unit17 capable of displaying data, etc. by using, for example, LCD and such,a processing unit 21 capable of executing various operations, acommunication control unit 23 capable of controlling the communicationsbetween the terminal and another equipment, an image capture unit 25capable of acquiring image data, etc., a position (including direction)verification unit 27 inluding a GPS and such, an information synthesisunit 31 capable of synthesizing various information, and a storage unit33 capable of storing data.

The input unit 15 has a character input unit 15 a and a speech inputunit 15 b. The processing unit 21 has a speech feature extraction unit21 a, a phoneme string generation unit 21 b and an image featureextraction unit 21 c.

The speech feature extraction unit 21 a extracts a feature from thespeech waveform of the speech inputted from the speech input unit 15 aand expresses the feature with numerical values for example, thereby toform a speech feature. The phoneme string generation unit 21 b generatesa phoneme string being the string of phonemes, on the basis of thespeech inputted from the speech input unit 15 a. The image featureextraction unit 21 c forms a feature of the image (such as a tendencyobtained from a certain image) on the basis of the image captured by theimage capture unit 25.

<Information Processing Device>

As shown in FIG. 4(A), the information processing device B such as aserver on the service provider side is constructed including an antenna40, a communication control unit 41, an information processing unit 47,and a storage unit 51.

The storage unit has a speech information storage area 53 for storinginformation on speech, a position information storage area 55 forstoring information on positions and directions, an image informationstorage area 57 for storing information on images captured, and anadditional information storage area 61 for storing additionalinformation relevant to these informations mentioned above.

Incidentally, FIG. 4(A) exemplifies an arrangement in which thecommunication control unit 41 and the information processing device 45are made unitary in the information processing device B, but the unit 41and the device 45 may well be separate as expressed by sign L1.

<Database>

The database as to which data registration, accumulation and retrievalare performed through the mobile information terminal A and theinformation processing device B may be constructed in the storage unit33 within the mobile information terminal A shown in FIG. 3, forexample. On the other hand, it may well be built in the storage unit 51within the information processing device B shown in FIG. 4(A), from theviewpoint that the information are provided or requested by a largenumber of users.

<System>

FIG. 4(B) shows a system configuration example in which databases DB,DB′ are separate from the mobile information terminals A, A′ beingterminal stations and from the information processing devices B, B′being base stations. Here, the pair of the mobile information terminal(terminal station) A—the information processing device (base station) Band the pair of the mobile information terminal (terminal station)A′—the information processing device (base station) B′ are respectivelyassociated with a network C. Further, the databases DB, DB′ areassociated with the information processing devices (base stations) B, B′through the network C. Of course, the system may well have any ofvarious other connection embodiments.

<<Concrete Description of Regisgtration and Retrieval Processes >>

Next, the process flow and the data structure examples of the databasein the information providing system as one embodiments of the inventionwill be described with reference to FIGS. 5 through 7. Also FIGS. 3 and4 will be referred to occasionally.

<Practicable Flow of Registration Process>

FIG. 5 is a practicable flowchart showing the process flow forregistering information in the database. Incidentally, the registrationprocess can also be called “database creation process”.

First, at a step S21, the database registration process is started. At astep S22, speech from the user of the mobile information terminal A isinputted by the speech input unit 15 b of the input unit 15 in themobile information terminal A. At a step S23, the inputted speech isanalyzed by the speech feature extraction unit 21 a of the processingunit 21, thereby to extract speech features or phoneme features. At astep S24, a phoneme string is generated from the extracted speechfeatures or phoneme features by the phoneme string generation unit 21 bof the processing unit 21.

Further, at a step S25, in accordance with the speech input, theposition of the mobile information terminal A is verified using a GPS orthe like by the position verification unit 27 (position/directiondetection step). The position information contains the information suchas a position specified with a latitude and longitude, a unique positionidentified by an address, etc. and a relative position as to whether ornot an object exists in which direction with respect to the positionreference of a certain specified building, etc. At a step S26, necessaryposition information is selected and specified from among the aboveinformation.

Further, at a step S27, an image is captured by the image capture unit25 in accordance with the speech input. At a step S28, an image featureis extracted from the captured image by the image feature extractionunit 21 c of the processing unit 21.

And, at a step S29, the phoneme string, the selected positioninformation and the image feature are associated by the informationsynthesis unit 31. In addition, relevant additional information which isrelevant to the phoneme string, the selected position information andthe image feature is created as indicated at a step S30. At a step S31,the phoneme string, the selected position information, the image featureand the relevant additional information are registered in the databasein a state where they are associated with one another. At a step S32,the process for the registration in the database is ended.

Structural examples of data registered in the database are shown in FIG.6. The speech information, position information, image information andadditional information are arranged in association. By the way, in theexamples shown in FIG. 6, each speech information is indicated byreading in katakana (the square form of the Japanese syllabarycharacters kana) for the sake of convenience, but they are actuallystored as phoneme strings by using the alphabet, phonetic symbols, orthe like.

The data structure of the first line indicates a case where the speechinformation is “Shibuya Department Store” (in actuality, an existentappellation is used), the position information is Shibuya Ward . . . andthe image information (feature) is the image of the Shibuya DepartmentStore. As the additional information, the information regarding therenewal sale at the Shibuya Department Store is listed, for example.

The data structure of the second line indicates a case where the speechinformation is “Hachi”, where the position information is Shibuya Ward .. . and the image information (feature) is the image of the Faithful DogHachiko. As the additional information, the map information of theHachiko Entrance of the Shibuya Station.

The data structure of the fifth line indicates a case where the speechinformation is “Kairakuen”, the position information is Mito City . . .and the image information (feature) is the image of ume (Japaneseapricot tree) garden”. As the additional information, the informationregarding the opening period of Ume Festival held at Ko⁻do⁻kan in MitoKairakuen.

As described above, the database in which the data relevant to oneanother are associated can be constructed on the basis of the speechinformation, the position information and the image information.

Incidentally, data at the third and fourth lines will be explainedlater.

The above processing will now be described more concretely.

For example, at Shibuya (a town in Japan), names regarding the objectscaught by cameras and the images and scenes in the lines of sight andsuch are spoken in the situation where a plurality of persons, eachcarrying a wearable computer equipped with a camera (image captureunit), a GPS (position verification units), etc., walks around and aplurality of wearable computers, each equipped with a camera, GPU, etc.and mounted on a wheeled chair, bicycle, etc., moves around and furtherin the situation where a plurality of radio controlled devices areoperated and the monitoring of nearby surroundings is performed while arobot cruises around.

For example, “Ha Chi” or “Ha Chi Ko” is pronounced according to thebronze statue of Hachhiko or “Shibuya Department Store” is pronounced infront of the department store. Then, basing on these speeches, speechfeature parameters, phonemic symbols, phonetic symbols, phonemic segmentsymbols, etc. which are in time series and the position information ofthe mobile information terminal A are mutually related and registered inthe database. On this occasion, image features which have been obtainedby the image feature extraction unit 21 c on the basis of imagescaptured by the image capture unit 25 and such may well be mutuallyrelated and stored in the database.

Subsequently, the above information is analyzed on the basis ofpositions and the symbol strings are statistically converged. Aprocessing method in the convergence can be easily constructed by knownstatistical techniques. For example, a method of calculating Euclideandistance based on means and variances, a similarity evaluation based onthe Bayes formula, a distance evaluation of VQ (Vector Quantity) codesafter the VQ coding based on the k-means clustering, or the like. As aresult, symbol strings converted from the speeches regarding the eventsseen at a certain position are statistically evaluated and then thesymbol strings converted from the speeches frequently spoken at thecertain postion are classified and stored on the basis of the positionsor directions. Accordingly, the information can be databased in detail.

In creating the database, the image features may be directly related andstored by image processing techniques, or the characters of a signboard,etc. may well be recognized and employed as the information of labelingand the recognized character string may expanded into phoneme strings,which are made utilizable in the information providing system of theinvention. Further, when the speech is expanded into phonemes and thenregistered in the database at the time of the information collection,there are some possibilities that several ways of reading (pronouncing)the Japanese written expressions such as “Shi Bu Ya Hya Kka Ten” and“Shi Bu Ya De Pa-To” for the expression “shibuyahyakkaten” might beinputted. Thus, considering these possibilities, the seeches of aplurality of users can be stored in the database It is also possible toexecute the processing in consideration of collateral information suchas the ages, genders or birthplaces of the speakers. Data may well bestatistically converged so as to absorb erroneous readings, changesascribable to dialects, the peculiar ways of speaking, the differencesof languages, etc. by executing a process in which pronounced phonemicseries having higher frequency in appearance is statistically deducedand then registered in the database. In these analyses, a more efficientanalysis and search can be realized by adding the information ofintonation or accent. Additionally, a still more efficient analysis andsearch can be realized by adding a learning function to the mobileinformation terminals A and B.

The creation of the database by the processing as stated abovecorresponds also to collecting regional information while speech labelsare being given to map information and ambient image features. Byanalyzing the collected information with the above statisticaltechniques based on the positions, the database of names and relevantterms of buildings etc. which come into user's sight during a walkthrough a certain region can be constructed. On this occasion, thecollected information can be related to various information based on notonly the phoneme strings, but also word character strings, speechwaveform features, speech VQ codes, etc. Alternatively, by appropriatelycombining the above elements, the accuracy of correspondence with theassociated information can be enhanced for the labeling.

Furthermore, in a case where a content concerning a position is to belabeled by inputting a word with a keyboard, input keys and such whilewatching an image, the word may well be expanded into time-series dataformed of the symbols of phonemes, phoneme segments or the likes, so asto employ the time-series data as a search label.

According to the database thus constructed, it is unnecessary to executea process in which speeches are converted into text labels containingwords and kanjis (Chinese characters used in Japanese writing) by speechrecognition. More specifically, since the phoneme strings arecontinuously outputted, it is permitted to evaluate and search for thephoneme strings one another by a technique such as DP matching or HMM(Hidden Markov Model), and appropriate information at the currentposition can be retrieved without identifying an accurate word.

Of course, a method is also suggested in which the similarities betweenphonemes are put together as a table for distance evaluationsbeforehand, thereby to enhance the efficiency of matching evaluations inDP (Dynamic Programming) matching. Further, the database may well beconstructed in such a way that several meaning or shape informationitems are given to a single name by adding a plurality of meaning tagsor image features to a single phoneme string, or that a plurality ofphoneme strings are related to a single meaning tag or shapeinformation. Thus, a semantic search can also be realized using a methodin which events relevant to an identical phoneme string tag areconverged as homonyms or different names for an identical shape, or inwhich phoneme strings relevant to the name of an identical shape or toan identical tag are set as synonyms, or in which a meaning tag itselfis formed of a phoneme string.

Further, in a case where an image feature having a phoneme stringdifferent from that of an image feature of high similarity is handled asa synonym, the database may well be constructed so that a meaning can beshared between different languages in consideration of the appearancerate, transition probability, etc. of phonemes attributed to thedifferent languages. Furthermore, a method is also possible in whichimages are evaluated as the difference of shapes based on the directionsof the lines of sight, from similar phonemic label series for differentimage features.

In the use of the database, a search for commerce-based objects such asstores can be made more efficiently in such a way that stores arepreviously classified by finer items such as the sorts of the stores,the open hours thereof, the genres thereof, and the list of stores in alarge-sized store. It is also allowed to simultaneously employinformation on the Internet, or to share information via the Internet.Further, it is possible to employ not only a terminal of portable orattachment type, such as cellular phone, wearable computer or carnavigation system, but also a device or robot having an informationprocessing function, such as pet robot or agent robot.

<Practicable Flow of Retrieval Process>

Next, the process in which relevant information is retrieved from adatabase based on speech and with information as a key will be describedwith reference to FIG. 7.

First, at a step S51, the retrieval process is started. At a step S52, auser of the mobile information terminal A, who makes a request forinformation provision, inputs speech relevant to the information he orshe requests by the speech input unit 15 b of the input unit 15 in themobile information terminal A. At a step S53, the speech analysis of theinputted speech is made by the speech feature extraction unit 21 a ofthe processing unit 21 so as to extract speech features or phonemefeatures. At a step S54, a phoneme string is generated on the basis ofthe extracted speech features or phoneme features by the phoneme stringgeneration unit 21 b of the processing unit 21.

Simultaneously with the above processing concerning the speech, at astep S55, the position of the mobile information terminal A is detectedby the position verification unit 27 in accordance with the speechinput. At a step S56, necessary position information is selected andspecified. In addition, at a step S57, an ambient environment iscaptured by the image capture unit 25 in accordance with the speechinput, and at a step S28, image features are extracted from the capturedimage by the image feature extraction unit 21 c of the processing unit21.

On the basis of the information items obtained at the above steps and inaccordance with an identical speech input, at a step S59, theinformation processing device B searches for the relevant informationfrom the database, affixes additional information thereto and sends theresulting information to the mobile information terminal A.

These relevant information items are displayed on, for example, thedisplay unit 17 of the mobile information terminal A. Then, the user canact with reference to the information displayed on the display unit 17.

A service which presents the above search result to the user may well,for example, display a graphical map or a route moving image, present aroute to a designated store, or indicate directions at importantpositions with voices. Further, an effective and efficient informationsupport can be implemented by making an artifice in which a person whoworks for a store or the customer of the store adds information such asan advertisement or evaluation, or in which information items arechanged-over between those for meals in the daytime and those forsaloons in the nighttime in accordance with the time zones of use.

Furthermore, as an integrated applicative example concerning the aboveservice, the user is permitted to go to the movie of “Faithful DogHachiko” in such a way that he/she has been specified to be in front of“Hachiko Statue” at Shibuya Station on the basis of his/her positioninformation via any of various techniques such as GPS and thereafterhe/she makes a multiple condition search employing keywords such as“Hachi” and “Video”. Also, it is possible to search for only sceneswhere the word of “Ha Chi” comes out in conversation within a movie orpictures with “Ha Chi Ko”.

Incidentally, referring to FIG. 6 again, even when the identical speechinformation of “Hachi” has been inputted, search results in the caseswhere the verified or specified positions are, for example, within thelimits of Mito City and in the vicinity of the traditional shopping andentertainment districts of Edo can become different.

For example, in a case where the position information has been specifiedas Mito City, the information of a popular television program which hasa character relevant to “Hachi” and which is relevant to “Mito City” maywell be preferentially added. Also, in a case where the positioninformation indicates old downtown of Tokyo, information relevant to aclassical comic story about that person of a tenement house of the poorin the “old downtown of Tokyo” who is relevant to “Hachi” is presented.

In this manner, since the cases of the same sound are confusing, theposition information is also considered, whereby the accuracy ofrecognition can be enhanced.

As exemplified by Mito Kairakuen mentioned above, it is also possible topresent guidance on noted places in various districts and informationrelevant to the noted places, and the information providing system canalso be employed as a guidebook for travelers. Further, the informationproviding system will be applicable to the guide of a disabled person,etc.

On this occasion, a more efficient search condition may well be set insuch a way that the relation of a plurality of words is subjected togrammatical regulations or a syntactic analysis by employing prior-artspeech recognition techniques.

Incidentally, regarding a method for detecting a position or direction,the position may well be detected just as in a car navigation system bycombining a GPS, a velocity sensor, a geomagnetic sensor or the likes,or it may well be detected on the basis of a received ID by setting tagsor the likes for transmitting IDs indicative of positions, in individualplaces.

Of course, the information items to be labeled are not limited tostores, signboards, timetables, guide maps, and monuments and the cityinformation items lying deep in regions, but they may well be, forexample, natural information items in the suburbs, such as growing treesor flowers, animals or insects. For example, in a case where the speechinformation is “O Ze”(a place in Japan), while the position informationindicates a direction toward Oze, relevant information such as a spotworthy of note for the flowers of a skunk cabbage at Oze may well beprovided. Further, in case of employing the natural information for thesearch, a technique as stated below is suggested.

First, conditions are set based on the frequency distribution, color orluminosity, and chroma saturation of a shape or contour which changesdepending upon natural situations such as season, weather and time zone,and also image information, acoustic information such as noise orambient sound, and the changes of natural concrete situations, such asthe changes of wind force, atmospheric temperature, rainfall andgeomagnetism. And then, an inputted ambient scene is classified intodata. Subsequently, the statistical deviation of features is analyzedusing feature information concerning an image or region.

In addition, this evaluation result can be used for labeling concerningthe natural information for the registered information, or it can beused for the revision or correction of transmission contents for theuser, considering the situation changes of weather in the search. Also,the user himself/herself can employ the result as a search key, or thesystem can utilize the result as a search key which is automaticallyinserted. Of course, significance tags based on IDs or phoneme stringsmay well be constructed on the basis of speeches which represent theinformation items, or the speeches may well be employed for constructinga phoneme database for investigating and researching into their speechfeatures.

In order to construct such a database, a person who inputs the labelsmay well be established as a professional person called “indexer” or“labeler”. Further, in performing the job as a profession, a labeler whoinputs lies, mistakes or false information may possibly pose a problemas to the payment of wages. The problem, however, can be solved bystatistically evaluating the reliabilities of the contents of utterancesand registrations by individual labelers, through the similarities ofspecified appellations, weather conditions or the like objectiveinformation items in, for example, an identical region, to those byother labelers, the information of evaluations for the labelinginformation by data users, the dispatch of reliable staff members fromthe side of the database construction, and the registered situation ofobjective information based on a map already registered.

Besides, the information inputting job can be performed while moving.Herein, if the names and situations of objects in the surroundings canbe spoken, a person can input information items relevant to a region,more efficiently than in case of inputting the information with akeyboard while he/she is watching a document such as past map, orsignboards in the actual place. Therefore, it is also possible toprovide a labor environment which is not affected by ages anddifferences in ability, including education and experience, and thedatabase can be realized by the working power of a worker who has beenunemployed for a long term, or a retired person. In addition, a wagesystem for labelers can adopt an auction-like method in which serviceefficiencies are rated on the basis of the above credibilities, and suchinformation users' evaluations that a labeler inputs information itemswhich are not noticed by other labelers, and that a labeler inputsinformation items which are often utilized. Besides, in a case where anyinformation is not registered on the side of a database or where a groupof exact object feature information items cannot be specified in theevaluation of a similarity or likelihood within the database, a labelermay well provide information for new labeling, in compliance with arequest from the database side. Of course, in order to obtain theinformation of an image, speech, position or the like, a search can alsobe made by mutually combining the various features of opticalinformation, a picture, an acoustic image, a shape, a color, a sound andthe likes, or by performing weighting based on statistical relevancies.

Of course, the present invention is not restricted to the foregoingembodiments, but various aspects are possible as to details.

INDUSTRIAL APPLICABILITY

As described above in detail, according to the present invention, thereare provided a novel information providing system capable to make thedatabase registration and database search of various informationincluding city information easy and highly-accurate for anybody, and amobile information terminal and an information processing device whichare used for the information providing system. Thus, the city databasecan be constructed by relating the laborious labeling job to the speechand position information, and enhancement in the convenience oflivelihood can also be attained by realizing information support forinhabitants.

1. An information providing system in which information to be registeredin a database is provided from a mobile information terminal to aninformation processing device, characterized in that: said mobileinformation terminal comprising: a speech input unit capable ofinputting speech as registration information during an informationregistration mode; a speech processing unit capable of acquiring speechinformation based on the speech inputted by said speech input unit; aposition verification unit capable of acquiring position information ofsaid mobile information terminal as registration information during theinformation registration mode; an additional information input unitcapable of inputting additional information relevant to the speechinformation and the position information as registration informationduring the information registration mode; and a transmitter unit capableof transmitting, to said information processing device, the speechinformation obtained by said speech processing unit, the positioninformation obtained by said position verification unit and theadditional information inputted by said additional information inputunit; and said information processing device comprising: a receiver unitcapable of receiving the speech information, the position informationand the additional information sent from said mobile informationterminal; and an information processing unit capable of mutuallyrelating and storing the speech information, the position informationand the additional information received by said receiver unit, in thedatabase.
 2. An information providing system in which informationregistered in a database is provided from an information processingdevice to a mobile information terminal having made a request forinformation provision, characterized in that: said mobile informationterminal comprising: a speech input unit capable of inputting speechrelevant to requested information as a search key during an informationprovision request mode; a speech processing unit capable of acquiringspeech information based on the speech inputted by said speech inputunit; a position verification unit capable of acquiring positioninformation of said mobile information terminal as a search key duringthe information provision request mode; a transmitter unit capable oftransmitting to said information processing device, the speechinformation obtained by said speech processing unit and the positioninformation obtained by said position verification unit; and a receiverunit capable of receiving additional information sent from saidinformation processing device, as the requested information; and saidinformation processing device comprising: a receiver unit capable ofreceiving the speech information and the position information sent fromsaid mobile information terminal; an information processing unit capableof retrieving the additional information relevant to the speechinformation and the position information received by said receiver unit,from the database having speech information, position information andadditional information mutually related and prestored therein; and atransmitter unit capable of transmitting the additional informationretrieved by said information processing unit, to said mobileinformation terminal.
 3. An information providing system in whichinformation to be registered in a database is provided from a mobileinformation terminal to an information processing device and informationregistered in the database is provided from said information processingdevice to a mobile information terminal having made a request forinformation provision, characterized in that: said mobile informationterminal comprising: a speech input unit capable of inputting speech asregistration information during an information registration mode, andcapable of inputting speech relevant to requested information as asearch key during an information provision request mode; a speechprocessing unit capable of acquiring speech information based on thespeech inputted by said speech input unit during the informationregistration mode, and capable of acquiring speech information based onthe speech inputted by said speech input unit during the informationprovision request mode; a position verification unit capable ofacquiring position information of said mobile information terminal asregistration information during the information registration mode, andcapable of acquiring position information of said mobile informationterminal as a search key during the information provision request mode;an additional information input unit capable of inputting additionalinformation relevant to the speech information and the positioninformation as registration information during the informationregistration mode; a transmitter unit capable of transmitting the speechinformation obtained by said speech processing unit, the positioninformation obtained by said position verification unit and theadditional information inputted by said additional information inputunit, to said information processing device during the informationregistration mode, and capable of transmitting the speech informationobtained by said speech processing unit and the position informationobtained by said position verification unit, to said informationprocessing device during the information provision request mode; and areceiver unit capable of receiving additional information sent from saidinformation processing device, as the requested information; and saidinformation processing device comprising: a receiver unit capable ofreceiving the speech information, the position information and theadditional information sent from said mobile information terminal duringthe information registration mode, and capable of receiving the speechinformation and the position information sent from said mobileinformation terminal during the information provision request mode; aninformation processing unit capable of mutually relating and storing thespeech information, the position information and the additionalinformation received by said receiver unit, in the database, during theinformation registration mode and also capable of retrieving theadditional information relevant to the speech information and theposition information received by said receiver unit, from the database,during the information provision request mode; and a transmitter unitcapable of transmitting the additional information retrieved by saidinformation processing unit, to said mobile information terminal.
 4. Aninformation providing system in which information to be registered in adatabase is provided from a mobile information terminal to aninformation processing device, characterized in that: said mobileinformation terminal comprising: a speech input unit capable ofinputting speech as registration information during an informationregistration mode; a speech processing unit capable of acquiring speechinformation based on the speech inputted by said speech input unit; aposition verification unit capable of acquiring position information ofsaid mobile information terminal as registration information during theinformation registration mode; an image capture unit capable ofcapturing an image as registration information during the informationregistration mode; an image processing unit capable of acquiring imageinformation based on the image captured by said image capture unit; anadditional information input unit capable of inputting additionalinformation relevant to the speech information, the position informationand the image information as registration information during theinformation registration mode; and a transmitter unit capable oftransmitting to said information processing device, the speechinformation obtained by said speech processing unit, the positioninformation obtained by said position verification unit, the imageinformation obtained by said image processing unit and the additionalinformation inputted by said additional information input unit; and saidinformation processing device comprising: a receiver unit capable ofreceiving the speech information, the position information, the imageinformation and the additional information sent from said mobileinformation terminal; and an information processing unit capable ofmutually relating and storing the speech information, the positioninformation, and the image information and the additional informationreceived by said receiver unit, in the database.
 5. An informationproviding system in which information registered in a database isprovided from an information providing device to a mobile informationterminal having made a request for information provision, characterizedin that: said mobile information terminal comprising: a speech inputunit capable of inputting speech relevant to requested information as asearch key during an information provision request mode; a speechprocessing unit capable of acquiring speech information based on thespeech inputted by said speech input unit; a position verification unitcapable of acquiring position information of said mobile informationterminal as a search key during the information provision request mode;an image capture unit capable of capturing an image as a search keyduring the information provision request mode; an image processing unitcapable of acquiring image information based on the image captured bysaid image capture unit; a transmitter unit capable of transmitting tosaid information processing device, the speech information obtained bysaid speech processing unit, the position information obtained by saidposition verification unit and the image information obtained by saidimage processing unit; and a receiver unit capable of receivingadditional information sent from said information processing device, asthe requested information; and said information processing devicecomprising: a receiver unit capable of receiving the speech information,the position information and the image information sent from said mobileinformation terminal; an information processing unit capable ofretrieving the additional information relevant to the speechinformation, the position information and the image information receivedby the receiver unit, from the database having speech information,position information, image information and additional informationmutually related and prestored therein; a transmitter unit capable oftransmitting the additional information retrieved by said informationprocessing unit, to said mobile information terminal.
 6. An informationproviding system in which information to be registered in a database isprovided from a mobile information terminal to an information processingdevice and information registered in the database is provided from saidinformation processing device to a mobile information terminal havingmade a request for information provision, characterized in that: saidmobile information terminal comprising: a speech input unit capable ofinputting speech as registration information during an informationregistration mode, and capable of inputting speech relevant to requestedinformation as a search key during an information provision requestmode; a speech processing unit capable of acquiring speech informationbased on the speech inputted by said speech input unit during theinformation registration mode, and capable of acquiring speechinformation based on the speech inputted by said speech input unitduring the information provision request mode; a position verificationunit capable of acquiring position information of said mobileinformation terminal as registration information during the informationregistration mode, and capable of acquiring position information of saidmobile information terminal as a search key during the informationprovision request mode; an image capture unit capable of capturing animage as registration information during the information registrationmode, and capable of capturing an image as a search key during theinformation provision request mode; an image processing unit capable ofacquiring image information based on the image captured by said imagecapture unit during the information registration mode, and capable ofacquiring image information based on the image captured by said imagecapture unit during the information provision request mode; anadditional information input unit capable of inputting additionalinformation relevant to the speech information, the position informationand the image information as registration information during theinformation registration mode; a transmitter unit capable oftransmitting the speech information obtained by said speech processingunit, the position information obtained by said position verificationunit, the image information obtained by said image processing unit andthe additional information inputted by said additional information inputunit, to said information processing device during the informationregistration mode, and capable of transmitting the speech informationobtained by said speech processing unit, the position informationobtained by said position verification unit and the image informationobtained by said image processing unit, to said information processingdevice during the information provision request mode; and a receiverunit capable of receiving additional information sent from saidinformation processing device, as the requested information; and saidinformation processing device comprising: a receiver unit capable ofreceiving the speech information, the position information, the imageinformation and the additional information sent from said mobileinformation terminal during the information registration mode, andcapable of receiving the speech information, the position informationand the image information sent from said mobile information terminalduring the information provision request mode; an information processingunit capable of mutually relating and storing the speech information,the position information, the image information and the additionalinformation received by said receiver unit, in the database, during theinformation registration mode, and capable of retrieving the additionalinformation relevant to the speech information, the position informationand the image information received by said receiver unit, from thedatabase, during the information provision request mode; and atransmitter unit capable of transmitting the additional informationretrieved by said information processing unit, to said mobileinformation terminal.
 7. An information providing system as in claim 1,characterized in that said speech information is information whichcontains a phoneme string of said speech.
 8. An information providingsystem as in claim 4, characterized in that said image information isfeatures of said image.
 9. A mobile information terminal which providesinformation to be registered in a database, to an information processingdevice, comprising: a speech input unit capable of inputting speech asregistration information during an information registration mode; aspeech processing unit capable of acquiring speech information based onthe speech inputted by said speech input unit; a position verificationunit capable of acquiring position information of said mobileinformation terminal as registration information during the informationregistration mode; an additional information input unit capable ofinputting additional information relevant to the speech information andthe position information as registration information during theinformation registration mode; and a transmitter unit capable oftransmitting the speech information obtained by said speech processingunit, the position information obtained by said position verificationunit and the additional information inputted by said additionalinformation input unit, to said information processing device capable ofmutually relating and storing them in the database.
 10. A mobileinformation terminal to which information registered in a database isprovided from an information processing device by making a request forinformation provision, comprising: a speech input unit capable ofinputting speech relevant to requested information as a search keyduring an information provision request mode; a speech processing unitcapable of acquiring speech information based on the speech inputted bysaid speech input unit; a position verification unit capable ofacquiring position information of said mobile information terminal as asearch key during the information provision request mode; a transmitterunit capable of transmitting to said information processing device, thespeech information obtained by said speech processing unit and theposition information obtained by said position verification unit; and areceiver unit capable of receiving as the requested information,additional information relevant to the speech information and theposition information retrieved and sent from the database having speechinformation, position information and additional information mutuallyrelated and prestored therein by said information processing device. 11.A mobile information terminal which provides information to beregistered in a database, to an information processing device,comprising: a speech input unit capable of inputting speech asregistration information during an information registration mode; aspeech processing unit capable of acquiring speech information based onthe speech inputted by said speech input unit; a position verificationunit capable of acquiring position information of said mobileinformation terminal as registration information during the informationregistration mode; an image capture unit capable of capturing an imageas registration information during the information registration mode; animage processing unit capable of acquiring image information based onthe image captured by said image capture unit; an additional informationinput unit capable of inputting additional information relevant to thespeech information, the position information and the image informationas registration information during the information registration mode;and a transmitter unit capable of transmitting the speech informationobtained by said speech processing unit, the position informationobtained by said position verification unit, the image informationobtained by said image processing unit and the additional informationinputted by said additional information input unit, to said informationprocessing device capable of mutually relating and storing them in thedatabase.
 12. A mobile information terminal which provides informationregistered in a database, from an information processing device to amobile information terminal having made a request for informationprovision, comprising: a speech input unit capable of inputting speechrelevant to requested information as a search key during an informationprovision request mode; a speech processing unit capable of acquiringspeech information based on the speech inputted by said speech inputunit; a position verification unit capable of acquiring positioninformation of said mobile information terminal as a search key duringthe information provision request mode; an image capture unit capable ofcapturing an image as a search key during the information provisionrequest mode; an image processing unit capable of acquiring imageinformation based on the image captured by said image capture unit; atransmitter unit capable of transmitting to said information processingdevice, the speech information obtained by said speech processing unit,the position information obtained by said position verification unit andthe image information obtained by said image processing unit; and areceiver unit capable of receiving as the requested information,additional information relevant to the speech information, the positioninformation and the image information retrieved and sent from thedatabase having speech information, position information, imageinformation and additional information mutually related and prestoredtherein by said information processing device.
 13. A mobile informationterminal as in claim 9, characterized in that said speech information isinformation which contains a phoneme string of said speech.
 14. A mobileinformation terminal as in claim 11, characterized in that said imageinformation is features of said image.
 15. An information processingdevice which registers information provided from a mobile informationterminal, in a database, comprising: a receiver unit capable ofreceiving, from said mobile information terminal, speech informationbased on speech inputted by said mobile information terminal, a positioninformation of said mobile information terminal, and additionalinformation relevant to the speech information and the positioninformation inputted by said mobile information terminal; and aninformation processing unit capable of mutually relating and storing thespeech information, the position information and the additionalinformation received by said receiver unit, in the database.
 16. Aninformation processing device which provides information registered in adatabase, to a mobile information terminal having made a request forinformation provision, comprising: a receiver unit capable of receiving,from said mobile information terminal, speech information based onspeech inputted by said mobile information terminal and a positioninformation of said mobile information terminal; an informationprocessing unit capable of retrieving additional information relevant tothe speech information and the position information received by saidreceiver unit, from the database having speech information, positioninformation and additional information mutually related and prestored asregistration information therein; and a transmitter unit capable oftransmitting the additional information retrieved by said informationprocessing unit, to said mobile information terminal.
 17. An informationprocessing device which registers information provided from a mobileinformation terminal, in a database, comprising: a receiver unit capableof receiving, from said mobile information terminal, speech informationbased on speech inputted by said mobile information terminal, a positioninformation of said mobile information terminal, image information basedon an image captured by said mobile information terminal, and additionalinformation relevant to the speech information, the position informationand the image information inputted by said mobile information terminal;and an information processing unit capable of mutually relating andstoring the speech information, the position information, the imageinformation and the additional information received by said receiverunit, in the database.
 18. An information processing device whichprovides information registered in a database, to a mobile informationterminal having made a request for information provision, comprising: areceiver unit capable of receiving, from said mobile informationterminal, speech information based on speech inputted by said mobileinformation terminal, a position information of said mobile informationterminal and image information based on an image captured by said mobileinformation terminal; an information processing unit capable ofretrieving additional information relevant to the speech information,the position information and the image information received by saidreceiver unit, from the database having speech information, positioninformation, image information and additional information mutuallyrelated and prestored therein; and a transmitter unit capable oftransmitting the additional information retrieved by said informationprocessing unit, to said mobile information terminal.
 19. An informationprocessing device as in claim 15, characterized in that said speechinformation is information which contains a phoneme string of saidspeech.
 20. An information processing device as in claim 17,characterized in that said image information is features of said image.21. An information providing system as in claim 2, characterized in thatsaid speech information is information which contains a phoneme stringof said speech.
 22. An information providing system as in claim 3,characterized in that said speech information is information whichcontains a phoneme string of said speech.
 23. An information providingsystem as in claim 4, characterized in that said speech information isinformation which contains a phoneme string of said speech.
 24. Aninformation providing system as in claim 5, characterized in that saidspeech information is information which contains a phoneme string ofsaid speech.
 25. An information providing system as in claim 6,characterized in that said speech information is information whichcontains a phoneme string of said speech.
 26. An information providingsystem as in claim 5, characterized in that said image information isfeatures of said image.
 27. An information providing system as in claim6, characterized in that said image information is features of saidimage.
 28. A mobile information terminal as in claim 10, characterizedin that said speech information is information which contains a phonemestring of said speech.
 29. A mobile information terminal as in claim 11,characterized in that said speech information is information whichcontains a phoneme string of said speech.
 30. A mobile informationterminal as in claim 12, characterized in that said speech informationis information which contains a phoneme string of said speech.
 31. Amobile information terminal as in claim 12, characterized in that saidimage information is features of said image.
 32. An informationprocessing device as in claim 16, characterized in that said speechinformation is information which contains a phoneme string of saidspeech.
 33. An information processing device as in claim 17,characterized in that said speech information is information whichcontains a phoneme string of said speech.
 34. An information processingdevice as in claim 18, characterized in that said speech information isinformation which contains a phoneme string of said speech.
 35. Aninformation processing device as in claim 18, characterized in that saidimage information is features of said image.